Integrative data analysis to further replicable science in psychology

Stefany Coxe

April 8, 2022

Hello

Applied statistics…

…in psychology

Replication crisis

Replication crisis

Why is it happening?

How can we do better?

Other methodological approaches

Integrative data analysis

Integrative data analysis (IDA)

Advantages of IDA

TIDAL dataset

TIDAL dataset

Data harmonization

Different measures

Data harmonization

Moderated nonlinear factor analysis

Moderated nonlinear factor analysis

Moderated nonlinear factor analysis

Moderated nonlinear factor analysis

Moderated nonlinear factor analysis

Moderated nonlinear factor analysis

Factor analysis versus item response theory

IRT item characteristic curve

Item characteristic curve

Item characteristic curve

Item characteristic curve

Parent depression: different measures

Depression symptoms

Symptom SCL-90R PHQ-9 QOL
Decreased interest or pleasure in activities 32 1 5
Depressed mood (sad, empty, hopeless) 20, 30, 54 2 26
Changes in sleep patterns 44, 64, 66 3 16
Fatigue or loss of energy 14 4 10
Change in weight and / or appetite 19 5 -
Feelings of worthlessness or excessive guilt 79 6 -
Diminished ability to concentrate 55 7 7
Psychomotor agitation - 8 -
Thoughts of death and / or suicide 15, 56 9 -

Getting to the final model

Moderated nonlinear factor analysis

Moderated nonlinear factor analysis

Item characteristic curves: reference study

Item characteristic curves: appetite and worthless (discrimination)

Item characteristic curves: bad sleep and worthless (difficulty)

Differences across studies

Item characteristic curves

Item characteristic curves: Study 3

Lower: trouble concentrating, suicidal thoughts, sleep problems
Higher: worthlessness, fatigue, lack of interest

Item characteristic curves: Study 4

Lower: fatigue
Higher: depressed mood, trouble concentrating

What now?

ADHD symptoms: different versions of the same measure

Differences across studies

Applied studies

Latent profile analysis

Latent profile analysis

Coxe, S., Sibley, M. H., & Becker, S. P. (2021). Presenting problem profiles for adolescents with ADHD: differences by sex, age, race, and family adversity. Child and Adolescent Mental Health, 26(3), 228-237.

Let’s do better science!

Thank you.

After credits…

Step 1: Consolidate items

Step 2: Factor analysis

Step 3a: 2PL IRT model

Step 3b: Covariate effects on factor mean and variance

Step 3c: Covariate effects on item loadings and intercepts

Final model

Table of results - depression

Symptom Threshold effect
Decreased interest or pleasure in activities 3
Depressed mood (sad, empty, hopeless) 4
Changes in sleep patterns 3
Fatigue or loss of energy 3, 4
Change in weight and / or appetite
Feelings of worthlessness or excessive guilt 3
Diminished ability to concentrate 3, 4
Thoughts of death and / or suicide 3

ADHD symptoms

Future directions

Using harmonized measure

Still investigating how the harmonized measures function